Image processing device and method, learning device and method, and program

ABSTRACT

There is provided an image processing device including a weight calculation unit that calculates a weight corresponding to each of a plurality of pixel values centering on a pixel of interest of an input image based on a feature amount calculated based on the plurality of pixel values centering on the pixel of interest, a regression coefficient reading unit that reads a regression coefficient stored for each class code determined based on a plurality of pixel values corresponding to the pixel of interest of the input image, and a pixel value calculation unit that calculates a pixel value of a pixel of interest of an output image by performing calculation using the plurality of pixel values, the weights, and the regression coefficients centering on the pixel of interest of the input image.

BACKGROUND

The present technology relates to an image processing device and method,a learning device and method, and a program, and more particularly, toan image processing device and method, a learning device and method, anda program capable of performing an image quality improvement process onvarious images including an edge or a texture without occurrence ofdeterioration.

In the past, in digital image data, sharpness processing has beenperformed to emphasize the contour or the like of an image. Byheightening a difference (contrast) between gray scales of a contourportion through the sharpness processing, the sense of sharpness of anappearance can be intensified.

However, when the sharpness processing is uniformly performed even onany image, image quality may actually deteriorate. For example, when thesharpness processing is performed on an artificial image such as acharacter or a simple figure, a portion in which an edge is originallysufficiently cleared through appropriate sharpness processing performedon a natural image may be overemphasized, and thus deterioration such asringing may occur in some cases.

That is, when sharpness of a texture portion in an image is attempted tobe improved through the sharpness processing, the deterioration such asringing may be conspicuous in an edge portion in the image. On the otherhand, when the process of suppressing the ringing on the edge portion inthe image is performed, the sharpness of the texture portion in theimage may be weakened.

Accordingly, the applicants have suggested a method of enabling animprovement in the sharpness of an edge and an improvement in thesharpness of a texture to be compatible (for example, see JapaneseUnexamined Patent Application Publication No. 2007-251689).

According to Japanese Unexamined Patent Application Publication No.2007-251689, for example, use of a dedicated edge process and adedicated texture process is sorted depending on a feature amount ofeach region in an image.

SUMMARY

In the technology of Japanese Unexamined Patent Application PublicationNo. 2007-251689, however, for example, a boundary between the dedicatededge process and the dedicated texture process may become prominent.

As in Japanese Unexamined Patent Application Publication No.2007-251689, for example, when the use of the dedicated edge process andthe dedicated texture process is sorted, it is necessary to implementtwo dedicated processes and implement a determination process of sortingthe use of the two dedicated processes. Therefore, there is a problemthat the scale of the processes may easily increase and high cost may beeasily caused.

Further, it is difficult to determine an edge portion and a textureportion in some cases. Therefore, when the process of improving thesharpness adaptively according to the image qualities of various inputimages is performed, there is a limit to suppression of the ringing atthe time of an attempt of considerable improvement in the sharpness.

It is desirable to provide a technology capable of performing an imagequality improvement process on various images including an edge or atexture without occurrence of deterioration.

According to a first embodiment of the present technology, there isprovided an image processing device including a weight calculation unitthat calculates a weight corresponding to each of a plurality of pixelvalues centering on a pixel of interest of an input image based on afeature amount calculated based on the plurality of pixel valuescentering on the pixel of interest, a regression coefficient readingunit that reads a regression coefficient stored for each class codedetermined based on a plurality of pixel values corresponding to thepixel of interest of the input image, and

a pixel value calculation unit that calculates a pixel value of a pixelof interest of an output image by performing calculation using theplurality of pixel values, the weights, and the regression coefficientscentering on the pixel of interest of the input image.

The weight calculation unit may calculate a feature amount having highcorrelation with presence of an edge based on the pixel values belongingto a group.

The image processing device may further include a group sorting unitthat sorts the plurality of pixel values centering on the pixel ofinterest of the input image into a plurality of groups. The weightcalculation unit calculates a weight corresponding to each of theplurality of pixel values centering on the pixel of interest, based onthe feature amount calculated for each group based on the pixel valuesbelonging to the group.

The weight calculation unit may sort a first number of pixel valuescentering on the pixel of interest into a plurality of groups which areconstituted by pixels at a plurality of positions including at least apixel position of the pixel of interest and are constituted by a secondnumber of pixel values, the second number being smaller than the firstnumber.

The regression coefficient may be learned and stored in advance. In thelearning, a high-quality image may be set as a teacher image and animage obtained by deteriorating the teacher image is set as a studentimage. A plurality of samples of a linear equation used to calculate apixel value of a pixel of interest of the teacher image using a pixelvalue of the student image and the weights may be generated for eachclass code, and the optimum regression coefficient is calculated fromthe plurality of samples.

According to the first embodiment of the present technology, there isprovided an image processing method including calculating, by a weightcalculation unit, a weight corresponding to each of a plurality of pixelvalues centering on a pixel of interest of an input image based on afeature amount calculated based on the plurality of pixel valuescentering on the pixel of interest, reading, by a regression coefficientreading unit, a regression coefficient stored for each class codedetermined based on a plurality of pixel values corresponding to thepixel of interest of the input image, and calculating, by a pixel valuecalculation unit, a pixel value of a pixel of interest of an outputimage through calculation of a linear equation using the plurality ofpixel values, the weights, and the regression coefficients centering onthe pixel of interest of the input image.

According to the first embodiment of the present technology, there isprovided a program for causing a computer to function as an imageprocessing device including a weight calculation unit that calculates aweight corresponding to each of a plurality of pixel values centering ona pixel of interest of an input image based on a feature amountcalculated based on the plurality of pixel values centering on the pixelof interest, a regression coefficient reading unit that reads aregression coefficient stored for each class code determined based on aplurality of pixel values corresponding to the pixel of interest of theinput image, and a pixel value calculation unit that calculates a pixelvalue of a pixel of interest of an output image by performingcalculation using the plurality of pixel values, the weights, and theregression coefficients centering on the pixel of interest of the inputimage.

According to the first embodiment of the present technology, a weightcorresponding to each of a plurality of pixel values centering on apixel of interest of an input image based on a feature amount calculatedbased on the plurality of pixel values centering on the pixel ofinterest may be calculated, a regression coefficient stored for eachclass code determined based on a plurality of pixel values correspondingto the pixel of interest of the input image is read, and a pixel valueof a pixel of interest of an output image by performing calculationusing the plurality of pixel values, the weights, and the regressioncoefficients centering on the pixel of interest of the input image maybe calculated.

According to a second embodiment of the present technology, there isprovided a learning device that sets a high-quality image as a teacherimage and sets an image obtained by deteriorating the teacher image as astudent image, the device including a weight calculation unit thatcalculates a weight corresponding to each of a plurality of pixel valuesof the student image based on a feature amount calculated based on theplurality of pixel values of the student image corresponding to a pixelof interest of the teacher image, and a regression coefficientcalculation unit that calculates a regression coefficient of regressionprediction calculation performed to calculate a pixel value of the pixelof interest of the teacher image through calculation of a linearequation using the plurality of pixel values, the weights, and theregression coefficients of the student image.

The weight calculation unit may calculate a feature amount having highcorrelation with presence of an edge based on the pixel values belongingto a group.

The learning device may further include a group sorting unit that sortsthe plurality of pixel values of the student image corresponding to thepixel of interest of the teacher image into a plurality of groups. Theweight calculation unit may calculate a weight corresponding to each ofthe plurality of pixel values of the student image, based on a featureamount calculated for each group based on the pixel values belonging tothe group.

The weight calculation unit may sort a first number of pixel valuescentering on the pixel of interest into a plurality of groups which areconstituted by pixels at a plurality of positions including at least apixel position of the pixel of interest and are constituted by a secondnumber of pixel values, the second number being smaller than the firstnumber.

According to the second embodiment of the present technology, there is alearning method that sets a high-quality image as a teacher image andsets an image obtained by deteriorating the teacher image as a studentimage, the method including calculating, by a weight calculation unit, aweight corresponding to each of a plurality of pixel values of thestudent image based on a feature amount calculated based on theplurality of pixel values of the student image corresponding to a pixelof interest of the teacher image, and calculating, by a regressioncoefficient calculation unit, a regression coefficient of regressionprediction calculation performed to calculate a pixel value of the pixelof interest of the teacher image through calculation of a linearequation using the plurality of pixel values, the weights, and theregression coefficients of the student image.

There is provided a program causing a computer to function as a learningdevice that sets a high-quality image as a teacher image and sets animage obtained by deteriorating the teacher image as a student image,the learning device including a weight calculation unit that calculatesa weight corresponding to each of a plurality of pixel values of thestudent image based on a feature amount calculated based on theplurality of pixel values of the student image corresponding to a pixelof interest of the teacher image, and a regression coefficientcalculation unit that calculates a regression coefficient of regressionprediction calculation performed to calculate a pixel value of the pixelof interest of the teacher image through calculation of a linearequation using the plurality of pixel values, the weights, and theregression coefficients of the student image.

According to the second embodiment of the present technology, a learningdevice sets a high-quality image as a teacher image and sets an imageobtained by deteriorating the teacher image as a student image, a weightcorresponding to each of a plurality of pixel values of the studentimage based on a feature amount calculated based on the plurality ofpixel values of the student image corresponding to a pixel of interestof the teacher image may be calculated, and a regression coefficient ofregression prediction calculation performed to calculate a pixel valueof the pixel of interest of the teacher image through calculation of alinear equation using the plurality of pixel values, the weights, andthe regression coefficients of the student image may be calculated.

According to the embodiments of the present technology, it is possibleto perform the image quality improvement process on various imagesincluding an edge or a texture without occurrence of deterioration.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an example of the configurationof a learning device according to an embodiment of the presenttechnology;

FIG. 2 is a diagram illustrating an example of a class code specifyingmethod;

FIG. 3 is a diagram illustrating an example of grouping of class taps;

FIG. 4 is a block diagram illustrating an example of the configurationof an image processing device according to an embodiment of the presenttechnology;

FIG. 5 is a flowchart illustrating an example of a learning process;

FIG. 6 is a flowchart illustrating an example of an image qualityimprovement process; and

FIG. 7 is a block diagram illustrating an example of the configurationof a personal computer.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

Hereinafter, an embodiment of the technology disclosed here will bedescribed with reference to the drawings.

FIG. 1 is a block diagram illustrating an example of the configurationof a learning device according to an embodiment of the presenttechnology.

A learning device 10 is configured as a learning device that is used foran image quality improvement process on an image and generates aregression coefficient used for the image quality improvement processbased on data of an input student image and an input teacher image (or ateacher signal). Here, the image quality improvement process is, forexample, a process of improving sharpness of an image or a process ofremoving noise of an image.

The learning device 10 is configured to learn regression coefficientswhich are coefficients used to generate a high-quality image close tothe teacher image as an output image by receiving the student image asan input image. As will be described in detail, for example, theregression coefficients are coefficients that are used for regressionprediction calculation of a linear equation used to calculate aprediction value of a pixel of interest in a quality-improved image bysetting feature amounts obtained from the values of a plurality ofpixels corresponding to a pixel of interest in an input image asparameters.

The regression coefficient is learned for each class code to bedescribed below and the regression coefficient is learned inconsideration of a weight to be described below.

The learning device 10 sorts a plurality of pixel values correspondingto a pixel of interest of an input image into one of a plurality ofclasses based on the plurality of pixel values corresponding to thepixel of interest and the feature amounts obtained from the pixelvalues. Further, the learning device 10 calculates weights by which thepixel values are to be multiplied based on the plurality of pixel valuescorresponding to the pixel of interest of the input image and thefeature amounts obtainable from the pixel values.

That is, in the image quality improvement process, the pixel of interestis sorted into a class based on the plurality of pixel valuescorresponding to the pixel of interest of the input image and thefeature amounts obtained from the pixel values. Then, by performingcalculation of the linear equation using the regression coefficient ofeach class learned by the learning device 10, the above-describedweights, and the plurality of pixel values corresponding to the pixel ofinterest of the input image, the pixel values of the quality-improvedimage are calculated.

The learning device 10 shown in FIG. 1 includes a class sorting unit 21,a weight calculation unit 22, a regression coefficient learning unit 23,and a regression coefficient memory 24.

In the learning device 10, for example, an image with no noise is inputas a teacher image and an image deteriorated in image quality by addingnoise to the teacher image is input as a student image. For example, animage obtained by adding various kinds of noise to the teacher image, animage obtained by adding a burr to the teacher image, an image obtainedby shifting a pixel position in the teacher image, an image obtained bythinning out a pixel in the teacher image, or the like is input as thestudent image.

The class sorting unit 21 extracts a class tap constituted by aplurality of pixel values of the student image corresponding to a pixelof interest of the teacher image and sorts the class tap into a classbased on a feature amount of the extracted class tap.

FIG. 2 is a diagram illustrating an example of a class sorting method.In this example, 9 pixel values corresponding to a pixel of interest ofthe input image and peripheral pixels of the pixel of interest areacquired as class taps. Further, identification numbers (tap numbers) 0to 8 are affixed in a preset order of the 9 pixel values.

Threshold values are determined for the pixel values of the tap numbers0 to 8 and code 1 or 0 is assigned based on the determination result ofthe threshold values. In this example, when the pixel value (luminancevalue) is greater than the threshold value, code 1 is assigned. When thepixel value (luminance value) is less than the threshold value, code 0is assigned.

Digits of binary numbers obtained by arranging the assigned codes of thetap numbers 0 to 8 are set as class codes. In this example, 100001011(267 in decimal) are set as the class codes. Thus, the class tap issorted into the class corresponding to the class codes 267. Further, theclass may be sorted in more detail using a difference or the likebetween the maximum and minimum values of the pixel values.

Specifying the class code shown in FIG. 2 is called 1 bit AdaptiveDynamic Range Coding (ADRC). Of course, the class sorting unit 21 mayspecify the class code according to a method other than the 1 bit ADRC.

Referring back to FIG. 1, the weight calculation unit 22 extracts aprediction tap from the plurality of pixel values of the student imagecorresponding to the pixel of interest of the teacher image andcalculates a weight corresponding to each pixel value in the predictiontap. The weight calculation unit 22 sorts the pixel values constitutingthe prediction tap into a plurality of groups and calculates a weightfor each group based on feature amounts obtained from the pixel valuesbelonging to each group. When ti is assumed to be the value of the pixelof interest of the teacher image, a plurality of pixel values xij of thestudent image corresponding to the pixel of interest of the teacherimage are extracted as the prediction tap.

FIG. 3 is a diagram illustrating grouping of the prediction taps. In thedrawing, 25 (=5×5) circles are illustrated and each circuit indicates apixel. The black circle in the middle of the drawing indicates a pixelof interest. That is, in the example of FIG. 3, the pixel values of 25pixels centering on the pixel of interest are set as a prediction tap.

In the example of FIG. 3, the pixel values constituting the predictiontap are sorted into four groups, groups G1 to G4. The group G1 is set asa group which is constituted by the pixel values of 9 pixels and inwhich the pixel of interest is disposed on the lower right side. Thegroup G2 is set as a group which is constituted by the pixel values of 9pixels and in which the pixel of interest is disposed on the lower leftside. The group G3 is set as a group which is constituted by the pixelvalues of 9 pixels and in which the pixel of interest is disposed on theupper right side. The group G4 is set as a group which is constituted bythe pixel values of 9 pixels and in which the pixel of interest isdisposed on the upper left side.

The weight calculation unit 22 calculates feature amounts having highcorrelation with presence of an edge based on the respective pixelvalues belonging to the groups G1 to G4. For example, feature amountssuch as dynamic ranges or activities are calculated as the featureamounts having the high correlation with the presence of the edge.

The weight calculation unit 22 specifies the maximum and minimum valuesof the feature amounts calculated for the groups G1 to G4, as describedabove, and calculates the weight set for each group. For example, aweight vg corresponding to a group Gg can be obtained as follows.

$\begin{matrix}{{Vg} = {f\left( {{\max\limits_{1 \leq m \leq 4}{param}_{m}}\; + {minparam}_{n} - {param}_{g}} \right)}} & (1)\end{matrix}$

In Expression (1), maxparam and miniparam indicate the maximum andminimum values of the feature amounts calculated for the groups G1 toG4. In Expression (1), the function f(x) is a function for adjusting aweight applying method. For example, a function such as Expression (2),Expression (3), or the like is used.

f(x)=1/(1+exp(−x))   (2)

f(x)=x ^(n)   (3)

A total sum of the weights by which the pixels in the prediction tap aremultiplied is normalized to be 1.0. That is, when L is assumed to be thetotal number of the pixel values of the prediction tap, the weight vijby which each (for example, 25) of the pixel values in the predictiontap is multiplied is calculated so that Expression (4) can be satisfied.Since the pixels at some positions near the pixel of interest in theprediction tap repeatedly belong to the plurality of groups, the valuesof these pixels are multiplied by the weight a plurality of times.

$\begin{matrix}{{\sum\limits_{j = 1}^{L}\; V_{ij}} = 1.0} & (4)\end{matrix}$

In Expression (4), suffix j indicates the identification number (tapnumber) of a pixel in the prediction tap.

The example shown in FIG. 3 is an example of the grouping and thegrouping may not necessarily be performed in this way. For example, thepixels arrayed in one line may be grouped. As the feature amounts havingthe high correlation with the presence of an edge, different featureamounts may be calculated for the individual groups. For example, thefeature amounts may be calculated such that a dynamic range iscalculated for the group G1, an activity is calculated for the group G2,and the like.

Referring back to FIG. 1, the regression coefficient learning unit 23learns the regression coefficient using the weights vij obtained asdescribed above. That is, the pixel of interest in the pixelsconstituting the teacher image are sequentially set and the weight vijis calculated in correspondence with each pixel of interest. Then,samples of a linear equation used to calculate the pixel values of theteacher image based on the pixel values of the student image aresequentially generated. The sample of the linear equation generated asdescribed above is added for each class code.

For example, when wcj is assumed to be a regression coefficientcorresponding to a class code c, the regression coefficient wcj may beobtained by the least-square method based on the samples of the linearequation added in the above-described way so that a minimizationfunction Q calculated by Expression (5) is the minimum. Thus, theoptimum regression coefficient wcj can be calculated when used incalculation of Expression (6) to be described below.

$\begin{matrix}{Q = {\sum\limits_{i = 1}^{N}\; \left\{ {t_{i} - {\sum\limits_{j = 1}^{L}\; {V_{ij}W_{cj}X_{ij}}}} \right\}^{2}}} & (5)\end{matrix}$

The regression coefficient wcj obtained in this way is stored in theregression coefficient memory 24 in correspondence with the class code.Thus, the regress coefficient is learned.

FIG. 4 is a block diagram illustrating an example of the configurationof an image processing device according to an embodiment of the presenttechnology. An image processing device 100 in the drawing is an imageprocessing device corresponding to the learning device 10 in FIG. 1.That is, the image processing device 100 performs regression predictioncalculation of the prediction tap obtained from an input image using theregression coefficients learned by the learning device 10 and performsimage processing to improve the quality of the input image.

The image processing device 100 shown in FIG. 4 includes a class sortingunit 121, a weight calculation unit 122, a prediction calculation unit123, and a regression coefficient memory 124.

The regression coefficients stored in the regression coefficient memory24 of the learning device 10 are stored in advance in the regressioncoefficient memory 124 of the image processing device 100.

The class sorting unit 121 extracts the class tap constituted by theplurality of pixel values corresponding to the pixel of interest of theinput image and sorts the class tap into a class based on the featureamounts of the extracted class tap. The class sorting performed by theclass sorting unit 121 of the image processing device 100 is performedaccording to the same method as that of the class sorting unit 21 of thelearning device 10. For example, as described above with reference toFIG. 2, the classes are sorted and the class code corresponding to theextracted class tap is specified.

The weight calculation unit 122 extracts the prediction tap constitutedby the plurality of pixel values corresponding to the pixel of interestof the input image and calculates a weight corresponding to each pixelvalue in the prediction tap. As described above with reference to FIG.3, as in the weight calculation unit 22 of the learning device 10, forexample, the weight calculation unit 122 of the image processing device100 sorts the pixel values constituting the prediction tap into aplurality of groups. Then, the weight calculation unit 122 calculates aweight for each group based on the feature amount obtained from thepixel values belonging to each group.

The feature amounts are set as feature amounts having high correlationwith presence of an edge. For example, feature amounts such as dynamicranges or activities are calculated as the feature amounts having thehigh correlation with the presence of the edge.

As described above, a weight vij by which each of the pixel values inthe prediction tap is multiplied is calculated.

The prediction calculation unit 123 reads the regression coefficient wcjcorresponding to the class code c specified by the class sorting unit121 from the regression coefficient memory 124.

The prediction calculation unit 123 multiplies the pixel valuesconstituting the prediction tap by the weights vij calculated by theweight calculation unit 122 and multiplies the pixel values constitutingthe prediction tap by the regression coefficient wcj read from theregression coefficient memory 124. That is, when i is assumed to be theidentification number of the pixel of interest, a prediction value yi ofthe pixel of interest generated from the input image is obtained by thelinear equation expressed by Expression (6).

$\begin{matrix}{y_{i} = {\sum\limits_{j = 1}^{L}\; {V_{ij}W_{cj}X_{ij}}}} & (6)\end{matrix}$

By repeating the calculation of Expression (6), the pixel value of eachof the pixels constituting an output image is calculated.

The image processing device 100 outputs the output image generated asdescribed above.

According to the embodiment of the present technology, as describedabove, the pixel values constituting the prediction tap are sorted inmore detail into groups and weights are calculated. Accordingly, animportance of an individual pixel in a reference region of the inputimage used to generate the pixels of the output image is adaptively setby a local feature amount. Specifically, the weights of the pixel valuesconstituting the prediction tap are switched according to the featureamount of each group.

In the embodiment of the present technology, the pixel values of theoutput image are calculated by applying the statistically optimizedcoefficients in addition to the setting of the weights for each class.

In the embodiment of the present technology, the importance of theindividual pixel in the reference region of the input image is set basedon the feature amount having the high correlation with the presence ofan edge. Accordingly, the pixels belonging to the group for which thereis a low probability of the presence of an edge are referred to morestrongly even when the pixels are the pixels in the same prediction tap.The pixels belonging to the group for which there is a high probabilityof the presence of an edge are referred to more weakly. This is becausethe pixels of a portion of an edge in an image are considered to havelow correlation with the peripheral pixels.

In the embodiment of the present technology, for example, byappropriately switching the weights of the pixels of the referenceregion without switching dedicated processes on an edge portion and atexture portion of an image, as in the related art, an appropriate imagequality improvement process can be performed on the edge portion or thetexture portion.

In the embodiment of the present technology, for example, the imagequality improvement process can be performed on the entire image withoutprominence in a process boundary between a dedicated edge process and adedicated texture process.

In the embodiment of the present technology, the sharpness of an edgecan be realized while suppressing deterioration such as ringing.Further, an improvement in the sharpness of a texture can besimultaneously realized. Compared to the related art, for example, theringing can be suppressed even when the sharpness is strongly improved.

In the embodiment of the present technology, since the regressioncoefficient used for the image quality improvement process is learnedfor each class code indicating the feature amount of the class tap inconsideration of the weight, the image quality improvement process canbe performed according to not only the edge or the texture but alsovarious characteristics.

In the embodiment of the present technology, the image qualityimprovement process can be performed on various images including an edgeor a texture without occurrence of deterioration.

Next, an example of a learning process performed by the learning device10 in FIG. 1 will be described with reference to the flowchart of FIG.5.

In step S21, the class sorting unit 21 extracts the class tapconstituted by the plurality of pixel values of the student imagecorresponding to the pixel of interest of the teacher image.

In step S22, the class sorting unit 21 sorts the class tap into theclass based on the feature amount of the class tap extracted in stepS21. At this time, for example, the class tap is sorted into the classaccording to the method described above with reference to FIG. 2.

In step S23, the weight calculation unit 22 extracts the prediction tapfrom the plurality of pixel values of the student image corresponding tothe pixel of interest of the teacher image.

In step S24, the weight calculation unit 22 sorts the pixel valuesconstituting the prediction tap into the plurality of groups. At thistime, as described above with reference to FIG. 3, for example, thepixel values constituting the prediction tap are sorted into theplurality of groups.

In step S25, the weight calculation unit 22 calculates the featureamount for each group sorted in the process of step S24. Here, thefeature amount is set as a feature amount having high correlation withpresence of an edge. For example, a feature amount such as a dynamicrange or an activity is calculated as the feature amount having the highcorrelation with the presence of the edge.

In step S26, the weight calculation unit 22 calculates the weightcorresponding to each pixel value in the prediction tap based on thefeature amount calculated in step S25. At this time, for example, asdescribed above, the weight vij by which each of the pixel values in theprediction tap is multiplied is calculated.

In step S27, the regression coefficient learning unit 23 generates thesamples of the linear equation used to calculate the pixel values of theteacher images based on the pixel values of the student image, using theweights vij calculated in the above-described way.

In step S28, the regression coefficient learning unit 23 adds thesamples generated in the process of step S27 for each class code.

In step S29, it is determined whether the addition corresponding to allthe samples ends. That is, it is determined whether the processes ofstep S21 to step S28 are performed with all of the pixels of the teacherimage set as the pixel of interest.

When it is determined in step S29 that the addition corresponding to allthe samples does not end, the process returns to step S21 and thesubsequent processes are repeatedly performed.

When it is determined in step S29 that the addition corresponding to allthe samples ends, the process proceeds to step S30.

In step S30, the regression coefficient learning unit 23 calculates theregression coefficient for each class code. At this time, for example,the regression coefficient wcj is obtained by the least-square methodbased on the samples of the linear equation added in the above-describedway so that the minimization function Q calculated by Expression (5) isthe minimum. The regression coefficient obtained here is stored in theregression coefficient memory 24 in correspondence with the class code.

In this way, the learning process is performed.

Next, an example of the image quality improvement process performed bythe image processing device 100 in FIG. 4 will be described withreference to the flowchart of FIG. 6. Before the process is performed,the regression coefficients calculated in the process of step S30 inFIG. 5 are assumed to be stored in the regression coefficient memory124.

In step S51, the class sorting unit 121 extracts the class tapconstituted by the plurality of pixel values corresponding to the pixelof interest of the input image.

In step S52, the class sorting unit 121 specifies the class code of theclass tap based on the feature amount of the class tap extracted in theprocess of step S51. The class sorting unit 121 of the image processingdevice 100 specifies the class code in step S52 according to the samemethod as that of the class sorting unit 21 of the learning device 10 instep S22. For example, as described above with reference to FIG. 2, theclass is sorted and the class code corresponding to the extracted classtap is specified.

In step S53, the weight calculation unit 122 extracts the prediction tapconstituted by the plurality of pixel values corresponding to the pixelof interest of the input image.

In step S54, the weight calculation unit 122 sorts the pixel valuesconstituting the prediction tap into the plurality of groups. At thistime, for example, as described above with reference to FIG. 3, thepixel values constituting the prediction tap are sorted into theplurality of groups.

In step S55, the weight calculation unit 122 calculates the featureamount for each group sorted in the process of step S54. Here, thefeature amount is set as a feature amount having high correlation withpresence of an edge. For example, a feature amount such as a dynamicrange or an activity is calculated as the feature amount having the highcorrelation with the presence of the edge.

In step S56, the weight calculation unit 122 calculates the weightcorresponding to each pixel value in the prediction tap based on thefeature amount calculated in step S55. At this time, for example, asdescribed above, the weight vij by which each of the pixel values in theprediction tap is multiplied is calculated.

In step S57, the prediction calculation unit 123 reads the regressioncoefficient wcj corresponding to the class code c specified by the classsorting unit 121 from the regression coefficient memory 124.

In step S58, the prediction calculation unit 123 calculates the pixelvalue by the linear equation using the weight vij obtained in theprocess of step S56 and the regression coefficient wcj read in theprocess of step S57. At this time, for example, each of the pixel valuesconstituting the prediction tap is multiplied by the weight vij and eachof the pixel values constituting the prediction tap is multiplied by theregression coefficient wcj. That is, the pixel value is obtained by thelinear equation expressed by Expression (6).

In step S59, it is determined whether calculation of the pixel valuescorresponding to all of the pixels of the output image ends.

When it is determined in step S59 that the calculation of the pixelvalues corresponding to all of the pixels of the output image does notend, the process returns to step S51 and the subsequent processes arerepeatedly performed.

When it is determined in step S59 that the calculation of the pixelvalues corresponding to all of the pixels of the output image ends, theprocess proceeds to step S60.

In step S60, the image processing device 100 outputs the output image.

In this way, the image quality improvement process is performed.

The above-described series of processes can be executed by hardware orsoftware. When the above-described series of processes is executed bythe software, a program constituting the software is installed from anetwork or a recording medium to a computer built in dedicated hardwareor a general-purpose personal computer 700, for example, which canexecute various functions by installing various programs, or the like,as illustrated in FIG. 7.

In FIG. 7, a central processing unit (CPU) 701 executes variousprocesses according to a program stored in a read only memory (ROM) 702or a program loaded from a storage unit 708 to a random access memory(RAM) 703. In addition, data or the like necessary for executing variousprocesses by the CPU 701 is appropriately stored in the RAM 703.

The CPU 701, the ROM 702, and the RAM 703 are connected to each othervia a bus 704. In addition, this bus 704 is also connected to aninput/output (I/O) interface 705.

An input unit 706 including a keyboard, a mouse or the like, an outputunit 707 including a display such as a liquid crystal display (LCD), aspeaker or the like, a storage unit 708 including a hard disk or thelike, and a communication unit 709 including a modem, a networkinterface card such as a local area network (LAN) card, or the like areconnected to the I/O interface 705. The communication unit 709 performsa communication process through a network such as the Internet.

If necessary, a drive 710 is connected to the I/O interface 705,removable media 711 such as a magnetic disk, an optical disc, amagneto-optical disc or a semiconductor memory are appropriatelymounted, and a computer program read therefrom is installed in thestorage unit 708, if necessary.

If the above-described series of processes is executed by software, aprogram constituting the software is installed from a network such asthe Internet or recording media including the removable media 711 andthe like.

This recording medium includes, for example, as illustrated in FIG. 7,the removable media 711 including a magnetic disk (including a floppydisk (registered trademark), an optical disc (including a compactdisc-read only memory (CD-ROM) or a digital versatile disc (DVD)), amagneto-optical disc (mini disc (MD) (registered trademark)), asemiconductor memory, or the like recording a program distributed to bedelivered to a user, separately from a device body. Also, the recordingmedium includes the ROM 702 recording a program to be delivered to auser in a state in which it is built in the device body in advance, or ahard disk included in the storage unit 708.

The series of processes described in this specification includesprocesses to be executed in time series in the described order andprocesses to be executed in parallel or individually, not necessarily intime series.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

Additionally, the present technology may also be configured as below.

-   -   (1) An image processing device including:

a weight calculation unit that calculates a weight corresponding to eachof a plurality of pixel values centering on a pixel of interest of aninput image based on a feature amount calculated based on the pluralityof pixel values centering on the pixel of interest;

a regression coefficient reading unit that reads a regressioncoefficient stored for each class code determined based on a pluralityof pixel values corresponding to the pixel of interest of the inputimage; and

a pixel value calculation unit that calculates a pixel value of a pixelof interest of an output image by performing calculation using theplurality of pixel values, the weights, and the regression coefficientscentering on the pixel of interest of the input image.

-   -   (2) The image processing device according to (1), wherein the        weight calculation unit calculates a feature amount having high        correlation with presence of an edge based on the pixel values        belonging to a group.    -   (3) The image processing device according to (1) or (2), further        including:

a group sorting unit that sorts the plurality of pixel values centeringon the pixel of interest of the input image into a plurality of groups,

wherein the weight calculation unit calculates a weight corresponding toeach of the plurality of pixel values centering on the pixel ofinterest, based on the feature amount calculated for each group based onthe pixel values belonging to the group.

-   -   (4) The image processing device according to (3), wherein the        weight calculation unit sorts a first number of pixel values        centering on the pixel of interest into a plurality of groups        which are constituted by pixels at a plurality of positions        including at least a pixel position of the pixel of interest and        are constituted by a second number of pixel values, the second        number being smaller than the first number.    -   (5) The image processing device according to any one of (1) to        (4),

wherein the regression coefficient is learned and stored in advance,

wherein, in the learning, a high-quality image is set as a teacher imageand an image obtained by deteriorating the teacher image is set as astudent image, and

wherein a plurality of samples of a linear equation used to calculate apixel value of a pixel of interest of the teacher image using a pixelvalue of the student image and the weights are generated for each classcode, and the optimum regression coefficient is calculated from theplurality of samples.

-   -   (6) An image processing method including:

calculating, by a weight calculation unit, a weight corresponding toeach of a plurality of pixel values centering on a pixel of interest ofan input image based on a feature amount calculated based on theplurality of pixel values centering on the pixel of interest;

reading, by a regression coefficient reading unit, a regressioncoefficient stored for each class code determined based on a pluralityof pixel values corresponding to the pixel of interest of the inputimage; and

calculating, by a pixel value calculation unit, a pixel value of a pixelof interest of an output image through calculation of a linear equationusing the plurality of pixel values, the weights, and the regressioncoefficients centering on the pixel of interest of the input image.

-   -   (7) A program for causing a computer to function as an image        processing device including:

a weight calculation unit that calculates a weight corresponding to eachof a plurality of pixel values centering on a pixel of interest of aninput image based on a feature amount calculated based on the pluralityof pixel values centering on the pixel of interest;

a regression coefficient reading unit that reads a regressioncoefficient stored for each class code determined based on a pluralityof pixel values corresponding to the pixel of interest of the inputimage; and

a pixel value calculation unit that calculates a pixel value of a pixelof interest of an output image by performing calculation using theplurality of pixel values, the weights, and the regression coefficientscentering on the pixel of interest of the input image.

-   -   (8) A learning device that sets a high-quality image as a        teacher image and sets an image obtained by deteriorating the        teacher image as a student image, the device including:

a weight calculation unit that calculates a weight corresponding to eachof a plurality of pixel values of the student image based on a featureamount calculated based on the plurality of pixel values of the studentimage corresponding to a pixel of interest of the teacher image; and

a regression coefficient calculation unit that calculates a regressioncoefficient of regression prediction calculation performed to calculatea pixel value of the pixel of interest of the teacher image throughcalculation of a linear equation using the plurality of pixel values,the weights, and the regression coefficients of the student image.

-   -   (9) The learning device according to (8), wherein the weight        calculation unit calculates a feature amount having high        correlation with presence of an edge based on the pixel values        belonging to a group.    -   (10) The learning device according to (8) or (9), further        including:

a group sorting unit that sorts the plurality of pixel values of thestudent image corresponding to the pixel of interest of the teacherimage into a plurality of groups,

wherein the weight calculation unit calculates a weight corresponding toeach of the plurality of pixel values of the student image, based on afeature amount calculated for each group based on the pixel valuesbelonging to the group.

-   -   (11) The learning device according to (10), wherein the weight        calculation unit sorts a first number of pixel values centering        on the pixel of interest into a plurality of groups which are        constituted by pixels at a plurality of positions including at        least a pixel position of the pixel of interest and are        constituted by a second number of pixel values, the second        number being smaller than the first number.    -   (12) A learning method of that sets a high-quality image as a        teacher image and sets an image obtained by deteriorating the        teacher image as a student image, the method including:

calculating, by a weight calculation unit, a weight corresponding toeach of a plurality of pixel values of the student image based on afeature amount calculated based on the plurality of pixel values of thestudent image corresponding to a pixel of interest of the teacher image;and

calculating, by a regression coefficient calculation unit, a regressioncoefficient of regression prediction calculation performed to calculatea pixel value of the pixel of interest of the teacher image throughcalculation of a linear equation using the plurality of pixel values,the weights, and the regression coefficients of the student image.

-   -   (13) A program causing a computer to function as a learning        device that sets a high-quality image as a teacher image and        sets an image obtained by deteriorating the teacher image as a        student image, the learning device including:

a weight calculation unit that calculates a weight corresponding to eachof a plurality of pixel values of the student image based on a featureamount calculated based on the plurality of pixel values of the studentimage corresponding to a pixel of interest of the teacher image; and

a regression coefficient calculation unit that calculates a regressioncoefficient of regression prediction calculation performed to calculatea pixel value of the pixel of interest of the teacher image throughcalculation of a linear equation using the plurality of pixel values,the weights, and the regression coefficients of the student image.

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2012-113275 filed in theJapan Patent Office on May 17, 2012, the entire content of which ishereby incorporated by reference.

What is claimed is:
 1. An image processing device comprising: a weight calculation unit that calculates a weight corresponding to each of a plurality of pixel values centering on a pixel of interest of an input image based on a feature amount calculated based on the plurality of pixel values centering on the pixel of interest; a regression coefficient reading unit that reads a regression coefficient stored for each class code determined based on a plurality of pixel values corresponding to the pixel of interest of the input image; and a pixel value calculation unit that calculates a pixel value of a pixel of interest of an output image by performing calculation using the plurality of pixel values, the weights, and the regression coefficients centering on the pixel of interest of the input image.
 2. The image processing device according to claim 1, wherein the weight calculation unit calculates a feature amount having high correlation with presence of an edge based on the pixel values belonging to a group.
 3. The image processing device according to claim 1, further comprising: a group sorting unit that sorts the plurality of pixel values centering on the pixel of interest of the input image into a plurality of groups, wherein the weight calculation unit calculates a weight corresponding to each of the plurality of pixel values centering on the pixel of interest, based on the feature amount calculated for each group based on the pixel values belonging to the group.
 4. The image processing device according to claim 3, wherein the weight calculation unit sorts a first number of pixel values centering on the pixel of interest into a plurality of groups which are constituted by pixels at a plurality of positions including at least a pixel position of the pixel of interest and are constituted by a second number of pixel values, the second number being smaller than the first number.
 5. The image processing device according to claim 1, wherein the regression coefficient is learned and stored in advance, wherein, in the learning, a high-quality image is set as a teacher image and an image obtained by deteriorating the teacher image is set as a student image, and wherein a plurality of samples of a linear equation used to calculate a pixel value of a pixel of interest of the teacher image using a pixel value of the student image and the weights are generated for each class code, and the optimum regression coefficient is calculated from the plurality of samples.
 6. An image processing method comprising: calculating, by a weight calculation unit, a weight corresponding to each of a plurality of pixel values centering on a pixel of interest of an input image based on a feature amount calculated based on the plurality of pixel values centering on the pixel of interest; reading, by a regression coefficient reading unit, a regression coefficient stored for each class code determined based on a plurality of pixel values corresponding to the pixel of interest of the input image; and calculating, by a pixel value calculation unit, a pixel value of a pixel of interest of an output image through calculation of a linear equation using the plurality of pixel values, the weights, and the regression coefficients centering on the pixel of interest of the input image.
 7. A program for causing a computer to function as an image processing device including: a weight calculation unit that calculates a weight corresponding to each of a plurality of pixel values centering on a pixel of interest of an input image based on a feature amount calculated based on the plurality of pixel values centering on the pixel of interest; a regression coefficient reading unit that reads a regression coefficient stored for each class code determined based on a plurality of pixel values corresponding to the pixel of interest of the input image; and a pixel value calculation unit that calculates a pixel value of a pixel of interest of an output image by performing calculation using the plurality of pixel values, the weights, and the regression coefficients centering on the pixel of interest of the input image.
 8. A learning device that sets a high-quality image as a teacher image and sets an image obtained by deteriorating the teacher image as a student image, the device comprising: a weight calculation unit that calculates a weight corresponding to each of a plurality of pixel values of the student image based on a feature amount calculated based on the plurality of pixel values of the student image corresponding to a pixel of interest of the teacher image; and a regression coefficient calculation unit that calculates a regression coefficient of regression prediction calculation performed to calculate a pixel value of the pixel of interest of the teacher image through calculation of a linear equation using the plurality of pixel values, the weights, and the regression coefficients of the student image.
 9. The learning device according to claim 8, wherein the weight calculation unit calculates a feature amount having high correlation with presence of an edge based on the pixel values belonging to a group.
 10. The learning device according to claim 8, further comprising: a group sorting unit that sorts the plurality of pixel values of the student image corresponding to the pixel of interest of the teacher image into a plurality of groups, wherein the weight calculation unit calculates a weight corresponding to each of the plurality of pixel values of the student image, based on a feature amount calculated for each group based on the pixel values belonging to the group.
 11. The learning device according to claim 10, wherein the weight calculation unit sorts a first number of pixel values centering on the pixel of interest into a plurality of groups which are constituted by pixels at a plurality of positions including at least a pixel position of the pixel of interest and are constituted by a second number of pixel values, the second number being smaller than the first number.
 12. A learning method of that sets a high-quality image as a teacher image and sets an image obtained by deteriorating the teacher image as a student image, the method comprising: calculating, by a weight calculation unit, a weight corresponding to each of a plurality of pixel values of the student image based on a feature amount calculated based on the plurality of pixel values of the student image corresponding to a pixel of interest of the teacher image; and calculating, by a regression coefficient calculation unit, a regression coefficient of regression prediction calculation performed to calculate a pixel value of the pixel of interest of the teacher image through calculation of a linear equation using the plurality of pixel values, the weights, and the regression coefficients of the student image.
 13. A program causing a computer to function as a learning device that sets a high-quality image as a teacher image and sets an image obtained by deteriorating the teacher image as a student image, the learning device including: a weight calculation unit that calculates a weight corresponding to each of a plurality of pixel values of the student image based on a feature amount calculated based on the plurality of pixel values of the student image corresponding to a pixel of interest of the teacher image; and a regression coefficient calculation unit that calculates a regression coefficient of regression prediction calculation performed to calculate a pixel value of the pixel of interest of the teacher image through calculation of a linear equation using the plurality of pixel values, the weights, and the regression coefficients of the student image. 